Skip to content

Keep compute logs in the URL, count filters across the window, and fix the Source filter - #634

Merged
mattdjenkinson merged 3 commits into
mainfrom
feat/logs-url-state
Nov 10, 2026
Merged

mattdjenkinson merged 3 commits into
mainfrom
feat/logs-url-state

Conversation

@mattdjenkinson

Copy link
Copy Markdown
Contributor

The instance and workload Logs tabs now keep their view in the URL and show filter counts for the whole time window. This also fixes the Source filter, which showed nothing for ALB, and makes the topology view and logs fill the page height.

Merge after production has milo-os/telemetry#267. The counts and the window-wide filter options use queryapi features from that PR. It is live on staging, but production pins o11y v0.6.8, which is older. Without it, counts are left off, but the HTTP filter options would list values from every ALB in the project instead of this one.

What changed

  1. Logs view in the URL. The time range, filters, search, live mode and the open row live in the query string (useComputeLogsUrlState, built on datum-ui's useLogsUrlState). A refresh or a shared link reopens the same view. The keys don't overlap the plugin's own timeRange, regions and instances params.
  2. Source and Severity filters.
    • Source = ALB showed "nothing found". Every ticked filter apart from Host went into the ALB LogQL, including Source and Severity. Source is a label the plugin adds in the browser, so the ALB query became {…, source="ALB"} and matched nothing. This was already on main.
    • Severity never filtered instance stdout.
    • The fix: only method and response_code go to the server now, and Severity filters stdout in the browser. When the filters rule a source out (Source, HTTP filters for stdout, Severity or Instance for ALB), that source's query is skipped.
  3. Filters for the whole window, with counts.
    • Options: Source, Severity, Method, Status code, Host and (on workloads) Instance come from label values for the whole window, not just the loaded rows.
    • Counts: each option shows how many lines match the other ticked filters, from sum by (…) (count_over_time(…)) queries. A filter whose rows the others hide shows zeros.
    • When counts hide: search and Host are applied in the browser, so counts hide while you search, and the Method, Status code and Source counts hide while a Host is ticked.
  4. Host shows the host each row shows. It uses the same order as the Host column (host, SNI, x_forwarded_host, then authority), so a backend name the proxy rewrote into authority (e.g. httpbin.org) is no longer an option.
  5. No loading flash on filter clicks.
    • Rows and filters: stay on screen while the next request runs, using datum-ui 3.11's isFetching state.
    • Counts: each count is its own useQuery, so the old numbers stay up until the new ones arrive. useQueries starts a fresh observer when a key changes and drops them.
  6. Full height.
    • Topology: the canvas grows to the bottom of the page, and refits as it resizes until you pan or zoom.
    • The rest: the page chrome, skeletons and logs cards flex to match.
  7. datum-ui 3.11.0 (chore(ui)).

Testing

  • tsc --noEmit passes, and pnpm install --frozen-lockfile works after the rebase onto main.
  • vitest: 426 tests pass. New tests cover:
    • the count queries for each combination of filters
    • turning counts into facet options, including zeros and failed queries
    • which sources still have rows to show
    • Source and Severity filtering
    • one host per row
  • Run locally through the portal (cloud-portal#1807) against staging.

The topology canvas grows to the bottom of the page instead of a fixed
frame, and refits as it resizes until you pan or zoom. The page chrome and
skeletons flex to match, so the logs tabs fill the height too.
…window

- Time range, filters, search, live and the open row live in the query
  string, so a refresh or shared link reopens the same view.
- Filter options come from label values for the whole window, and each
  shows how many lines match the other ticked filters (queryapi's sum by
  count_over_time), so ticking one never hides the others.
- Source and Severity no longer go into the ALB LogQL. Source is a label
  the plugin adds, so Source = ALB matched no ALB rows; Severity now filters
  instance stdout. A query whose rows the filters rule out is skipped.
- Host lists the host each row shows, not every host label on it.
- Earlier rows stay on screen while the next request runs, with datum-ui's
  isFetching state.
@mattdjenkinson mattdjenkinson added the enhancement New feature or request label Nov 10, 2026
@mattdjenkinson
mattdjenkinson requested a review from a team as a code owner November 10, 2026 11:05
@mattdjenkinson mattdjenkinson added the enhancement New feature or request label Nov 10, 2026
@mattdjenkinson mattdjenkinson self-assigned this Nov 10, 2026
mattdjenkinson added a commit to datum-cloud/cloud-portal that referenced this pull request Nov 10, 2026
…cal sign-in (#1807)

ALB logs now keep their view in the URL and show filter counts for the
whole time window. This also fixes the local sign-in 500, a toolbar that
jumped while the metrics page loaded, and logs pages that scrolled past
the bottom.

**Merge after production has milo-os/telemetry#267.** The counts and the
window-wide filter options use queryapi features from that PR. It is
live on staging, but production pins o11y `v0.6.8`, which is older.
Without it, counts are left off, but the filter options would list
values from every ALB in the project instead of this one.

## What changed

1. **Logs view in the URL.** The time range, filters, search, live mode
and the open row live in the query string, so a refresh or a shared link
reopens the same view. Refresh slides a preset window like "last 30
minutes" up to now. This uses `useLogsUrlState` from datum-ui.
2. **Filters for the whole window, with counts.**
- **Options:** come from label values for the whole window, not just the
500 loaded rows. Ticking one option no longer hides the others.
- **Counts:** each option shows how many lines match the other ticked
filters, from one `sum by (…) (count_over_time(…))` query per filter. So
ticking GET still shows how many POSTs there are.
- **When counts hide:** search and Host are applied in the browser, so
counts hide while you search. Method and Status code counts also hide
while a Host is ticked.
3. **Host shows the host each row shows.** The Host filter used every
host label on a row, including `authority`, which the proxy rewrites to
the backend's name. So `httpbin.org` was an option even though no row
showed it. Host now uses the same order as the Host column (`host`, SNI,
`x_forwarded_host`, then `authority`), and empty `-` values are skipped.
4. **No loading flash on filter clicks.** Earlier rows and filters stay
on screen while the next request runs, using datum-ui 3.11's
`isFetching` state: dimmed rows, a spinning Refresh and a thin progress
bar. Filter groups start open.
5. **Sign-in 500 on Node 26** (`fix(server)`).
- **The error:** Node 26 throws "Illegal invocation" when undici copies
`@hono/node-server`'s lightweight `Request`, which remix-auth does in
login, reauth and the OAuth callback.
- **Why production was fine:** the forwarded-proto middleware happens to
rebuild every request behind the gateway. Locally nothing did.
- **The fix:** a global middleware now converts each request once,
before anything else reads it.
6. **Metrics toolbar** (`fix(metrics)`).
- **No reflow on load:** the Step and Auto-refresh selects render their
label on the server. Radix otherwise fills it in on hydration and the
toolbar jumped.
- **Refresh corners:** the button squares its own right corners, because
datum-ui's Tooltip wrapper stops ButtonGroup from reaching it.
- **Alignment:** filter multi-selects use datum-ui's `MultiSelect` so
they line up with the other controls.
7. **Full-height pages** (`fix(layout)`). SubLayout's content was
`h-full` of the whole layout, title and tabs included, so logs ran past
the bottom. It now takes the remaining space with `flex-1`.
8. **datum-ui 3.11.0** (`chore(deps)`). Brings in `isFetching`, filter
groups open by default, and the picker clear button that no longer sits
inside the trigger button.

## Testing

- `npm run typecheck`, `eslint` and the pre-commit hook pass.
- The `app/resources/o11y-logs`, `app/features/edge` and `app/server`
tests pass (403 tests). New tests cover:
  - building the count queries
  - when counts are shown
  - one host per row
  - reading the count response
  - native requests
- Run locally against staging: sign-in works, and the ALB logs page
showed the right counts per filter (GET 9, 200 9, one host with 9).

The compute plugin's logs get the same changes in
datum-cloud/compute#634.

@ecv ecv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Every query goes through the project-scoped path, so until production has milo-os/telemetry#267 the extra filter options come from other ALBs in the same project, never another project. The lockfile moves datum-ui alone.

  • Counts turn off on the older queryapi only because it answers the metric query with a 400 and the client does not retry a 400. There is no feature check, so if production answered 200 with an empty result the facets would show zeros. One request against production before merge settles it.

Follow-ups:

  • withLabelMatcher escapes a multi-value filter with the regexp escape only (o11y-logs.ts:955), which leaves " unescaped, and buildComputeLogQLMany does the same. A hand-edited link with a " in a Severity or Instance value breaks the quoted matcher, which errors or adds a matcher within the user's own project. Apply the quote escape after the regexp escape, and add a test with " and \ in a multi-value filter.

@mattdjenkinson
mattdjenkinson merged commit e350a65 into main Nov 10, 2026
14 checks passed
@mattdjenkinson
mattdjenkinson deleted the feat/logs-url-state branch November 10, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants